Facilitating Integration of Distributed Statistical Databases Using Metadata and XML

In a distributed statistical database environment, data selection and actual statistical computation can be carried out at local databases, while data integration can bring together data produced from diverse sources, at different levels of details to produce statistical summaries. For example, the amount of beef retailed in European countries may be held in different local consumer databases and brought each country’s consumer index. Then the total amount of beef retailed across the countries may be compared and summarized. Such data integration often requires associated local (data provider) and global (domain) metadata and can be treated as a process of distributed database querying, which is accomplished by SQL native operators or the external statistical operators of combining SELECT and GROUPBY. In this paper, we present an integration approach that was developed in a fourth framework project, ADDSIA (Access to Distributed Databases for Statistical Information and Analysis), and which is built on a semi-structured data framework consisting of local and global metadata coded in XML [1, 2]. Extensions of this approach will be used in a fifth framework project, MISSION (Multi-Agent Integration of Shared Statistical Information Over the [inter]Net), in which the mapping between the local metadata and public metadata standards will be dynamically established through agent Abstract: This paper describes a novel approach for integrating distributed databases. It is based on a semi-structured data framework which uses XML (eXtensible Markup Language) and metadata derived from data productions and corresponding databases. This framework serves as an "integrated dynamic content table", enabling the user to browse the relevant information and database structure required prior to query composition; it provides a robust facility for sending a valid query to distributed databases without a prior knowledge of the component schema structure; a new class of statistical applications may therefore be easily built and managed. It remedies deficiencies in querying to distributed databases that heavily relies on prior understanding of the schema structure. An initial prototype has been developed.